{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# **Why function calling?**\n",
        "\n",
        "Imagine asking someone to write down important information without giving them a form or any guidelines on the structure. You might get a beautifully crafted paragraph, but extracting specific details like names, dates, or numbers would be tedious! Similarly, trying to get consistent structured data from a generative text model without function calling can be frustrating. You're stuck explicitly prompting for things like JSON output, often with inconsistent and frustrating results.\n",
        "\n",
        "This is where function calling comes in. Instead of hoping for the best in a freeform text response from a generative model, you can define clear functions with specific parameters and data types. These function declarations act as structured guidelines, guiding the Llama model to structure its output in a predictable and usable way. No more parsing text responses for important information!\n",
        "\n",
        "Think of it like teaching Llama to speak the language of your applications. Need to retrieve information from a database? Define a search_db function with parameters for search terms. Want to integrate with a weather API? Create a get_weather function that takes a location as input. Function calling bridges the gap between human language and the structured data needed to interact with external systems.\n",
        "\n",
        "# **Objectives**\n",
        "\n",
        "In this tutorial, you will learn how to use either OpenAI SDK or Vertex AI SDK in Python to make function calls via the Llama 4 Maverick fully managed model on Vertex AI.\n",
        " See here for more info on using the [OpenAI SDK with Vertex](https://cloud.google.com/vertex-ai/generative-ai/docs/migrate/openai/overview#:~:text=The%20Chat%20Completions%20API%20works,the%20Google%20Gen%20AI%20SDK.), as well as recommendations on when to use OpenAI SDK vs. Vertex AI SDK.\n",
        "\n",
        "We will use a currency exchange function as an example, you can replace it with another function with the right functionality for you.\n",
        "This tutorial is based on this Vertex AI codelab: https://codelabs.developers.google.com/codelabs/gemini-function-calling\n",
        "\n",
        "# **Setup and Relevant Links**\n",
        "Llama on Vertex AI (fully managed): https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/llama\n",
        "\n",
        "Official docs from Vertex on tool calling with Llama coming soon.\n"
      ],
      "metadata": {
        "id": "JIVs8gXB1Auw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **First, with OpenAI's SDK**"
      ],
      "metadata": {
        "id": "1zikcmKb5JVx"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Handling imports and setup"
      ],
      "metadata": {
        "id": "8znWkSqT5zyy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import google.auth\n",
        "import openai\n",
        "import json\n",
        "\n",
        "from google.auth.transport import requests as google_requests"
      ],
      "metadata": {
        "id": "C81B_THj5XUZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "\n",
        "project_id = \"[your-project-id]\"  # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
        "if not project_id or project_id == \"[your-project-id]\":\n",
        "    project_id = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
        "\n",
        "location = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-east5\")\n",
        "\n",
        "# run gcloud auth print-access-token from terminal to get this\n",
        "access_token = \"\"\n",
        "\n",
        "# Set up OpenAI client\n",
        "base_url = f\"https://{location}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{location}/endpoints/openapi\"\n",
        "client = openai.OpenAI(base_url=base_url, api_key=access_token)"
      ],
      "metadata": {
        "id": "ivcFdple7Qr5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "define the function implementation"
      ],
      "metadata": {
        "id": "wqhyLJO07Gaz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def get_exchange_rate(currency_date, currency_from, currency_to):\n",
        "    \"\"\"Get the exchange rate for currencies between countries\"\"\"\n",
        "    try:\n",
        "        url = f\"https://api.frankfurter.app/{currency_date}\"\n",
        "        params = {\n",
        "            \"from\": currency_from,\n",
        "            \"to\": currency_to\n",
        "        }\n",
        "        response = requests.get(url, params=params)\n",
        "        response.raise_for_status()\n",
        "        return json.dumps(response.json())\n",
        "    except Exception as e:\n",
        "        return f\"Error fetching exchange rate: {str(e)}\""
      ],
      "metadata": {
        "id": "4JagRT1t7DT8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "set up the function declaration for the model"
      ],
      "metadata": {
        "id": "LmS_8mdenPEU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tools = [{\n",
        "    \"type\": \"function\",\n",
        "    \"function\": {\n",
        "        \"name\": \"get_exchange_rate\",\n",
        "        \"description\": \"Get the exchange rate for currencies between countries\",\n",
        "        \"parameters\": {\n",
        "            \"type\": \"object\",\n",
        "            \"properties\": {\n",
        "                \"currency_date\": {\n",
        "                    \"type\": \"string\",\n",
        "                    \"description\": \"A date in YYYY-MM-DD format or 'latest'\"\n",
        "                },\n",
        "                \"currency_from\": {\n",
        "                    \"type\": \"string\",\n",
        "                    \"description\": \"The currency to convert from in ISO 4217 format\"\n",
        "                },\n",
        "                \"currency_to\": {\n",
        "                    \"type\": \"string\",\n",
        "                    \"description\": \"The currency to convert to in ISO 4217 format\"\n",
        "                }\n",
        "            },\n",
        "            \"required\": [\"currency_from\", \"currency_to\"]\n",
        "        }\n",
        "    }\n",
        "}]"
      ],
      "metadata": {
        "id": "4lP5F51Cmq7f"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "set up the client, generate a tool call and execute the tool call\n",
        "Enter a query into the text field to get started, eg. \"100 usd to eur\""
      ],
      "metadata": {
        "id": "iCGAtoNwmlFc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def main():\n",
        "    # Set up client\n",
        "    client = openai.OpenAI(base_url=base_url, api_key=access_token)\n",
        "\n",
        "    # Step 1: Send the user query to the model\n",
        "    user_query = input(\"Enter your currency exchange query: \")\n",
        "\n",
        "    completion = client.chat.completions.create(\n",
        "        model=\"meta/llama-4-maverick-17b-128e-instruct-maas\",\n",
        "        messages=messages,\n",
        "        tools=tools,\n",
        "        temperature=0.1,\n",
        "        max_tokens=512,\n",
        "        tool_choice=\"auto\"\n",
        "    )\n",
        "\n",
        "    # Step 2: Check if the model called a function\n",
        "    if completion.choices[0].message.tool_calls:\n",
        "        # Step 3: Execute the function\n",
        "        for tool_call in completion.choices[0].message.tool_calls:\n",
        "            if tool_call.function.name == \"get_exchange_rate\":\n",
        "                args = json.loads(tool_call.function.arguments)\n",
        "                currency_date = args.get(\"currency_date\", \"latest\")\n",
        "                currency_from = args.get(\"currency_from\")\n",
        "                currency_to = args.get(\"currency_to\")\n",
        "                result = get_exchange_rate(currency_date, currency_from, currency_to)\n",
        "\n",
        "                # Add model's response with function call\n",
        "                messages.append(completion.choices[0].message)\n",
        "                messages.append({\n",
        "                    \"role\": \"tool\",\n",
        "                    \"tool_call_id\": tool_call.id,\n",
        "                    \"content\": result\n",
        "                })\n",
        "\n",
        "                # Get final response from model\n",
        "                final_response = client.chat.completions.create(\n",
        "                    model=\"meta/llama-4-maverick-17b-128e-instruct-maas\",\n",
        "                    messages=messages,\n",
        "                    temperature=0.1\n",
        "                )\n",
        "\n",
        "                print(\"Final response:\", final_response.choices[0].message.content)\n",
        "    else:\n",
        "        print(\"Direct response:\", completion.choices[0].message.content)\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    main()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SycilP3AkiRr",
        "outputId": "f3bcab82-9010-4610-d392-f36792c57844"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Enter your currency exchange query: 100 usd to eur\n",
            "Direct response: The exchange rate for 100 USD to EUR is 88.645 EUR.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Now with Vertex SDK**"
      ],
      "metadata": {
        "id": "kyp2Fy_7noaV"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Handling imports and setup"
      ],
      "metadata": {
        "id": "KucVr8Txplnz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import json\n",
        "import requests\n",
        "import vertexai\n",
        "from vertexai.generative_models import (\n",
        "    FunctionDeclaration,\n",
        "    GenerationConfig,\n",
        "    GenerativeModel,\n",
        "    Tool,\n",
        "    Content,\n",
        "    Part\n",
        ")\n",
        "\n",
        "# Authenticate with Google Cloud\n",
        "from google.colab import auth\n",
        "auth.authenticate_user()"
      ],
      "metadata": {
        "id": "JxVuZ3R_pmKH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Initialize Vertex AI\n",
        "import os\n",
        "\n",
        "PROJECT_ID = \"[your-project-id]\"  # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
        "if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
        "    PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
        "\n",
        "LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\")\n",
        "\n",
        "vertexai.init(project=PROJECT_ID, location=LOCATION)"
      ],
      "metadata": {
        "id": "r1rP7PdUpwK6"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "define the function implementation"
      ],
      "metadata": {
        "id": "nFFEsV1BptAD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def get_exchange_rate(currency_date, currency_from, currency_to):\n",
        "    \"\"\"Get the exchange rate for currencies between countries\"\"\"\n",
        "    try:\n",
        "        url = f\"https://api.frankfurter.app/{currency_date}\"\n",
        "        params = {\n",
        "            \"from\": currency_from,\n",
        "            \"to\": currency_to\n",
        "        }\n",
        "        response = requests.get(url, params=params)\n",
        "        response.raise_for_status()\n",
        "        return response.json()\n",
        "    except Exception as e:\n",
        "        return {\"error\": str(e)}"
      ],
      "metadata": {
        "id": "hDMzxUBxptq0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "set up the function declaration for the model"
      ],
      "metadata": {
        "id": "g1RH1tplqhi5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "get_exchange_rate_func = FunctionDeclaration(\n",
        "    name=\"get_exchange_rate\",\n",
        "    description=\"Get the exchange rate for currencies between countries\",\n",
        "    parameters={\n",
        "        \"type\": \"object\",\n",
        "        \"properties\": {\n",
        "            \"currency_date\": {\n",
        "                \"type\": \"string\",\n",
        "                \"description\": \"A date in YYYY-MM-DD format or 'latest'\"\n",
        "            },\n",
        "            \"currency_from\": {\n",
        "                \"type\": \"string\",\n",
        "                \"description\": \"The currency to convert from in ISO 4217 format\"\n",
        "            },\n",
        "            \"currency_to\": {\n",
        "                \"type\": \"string\",\n",
        "                \"description\": \"The currency to convert to in ISO 4217 format\"\n",
        "            }\n",
        "        },\n",
        "        \"required\": [\"currency_from\", \"currency_to\"]\n",
        "    },\n",
        ")\n",
        "\n",
        "exchange_tool = Tool(\n",
        "    function_declarations=[get_exchange_rate_func],\n",
        ")\n"
      ],
      "metadata": {
        "id": "XyR1QzkSozrq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "set up the client, generate a tool call and execute the tool call\n",
        "Enter a query into the text field to get started, eg. \"100 usd to eur\""
      ],
      "metadata": {
        "id": "MFM6gfqwqzsi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model = GenerativeModel(\n",
        "    \"llama-4-maverick-17b-128e-instruct-maas\",\n",
        "    generation_config=GenerationConfig(temperature=0.1),\n",
        "    tools=[exchange_tool],\n",
        ")\n",
        "\n",
        "def main():\n",
        "    user_query = input(\"Enter your currency exchange query: \")\n",
        "\n",
        "    # Start chat and send user query\n",
        "    chat = model.start_chat()\n",
        "    response = chat.send_message(user_query)\n",
        "\n",
        "    # Check for function calls\n",
        "    if response.candidates[0].function_calls:\n",
        "        for function_call in response.candidates[0].function_calls:\n",
        "            if function_call.name == \"get_exchange_rate\":\n",
        "                # Extract arguments\n",
        "                args = function_call.args\n",
        "                currency_date = args.get(\"currency_date\", \"latest\")\n",
        "                currency_from = args.get(\"currency_from\")\n",
        "                currency_to = args.get(\"currency_to\")\n",
        "\n",
        "                # Call the function\n",
        "                result = get_exchange_rate(currency_date, currency_from, currency_to)\n",
        "\n",
        "                # Send function result back to model\n",
        "                final_response = chat.send_message(\n",
        "                    Content(\n",
        "                        role=\"function\",\n",
        "                        parts=[\n",
        "                            Part.from_function_response(\n",
        "                                name=\"get_exchange_rate\",\n",
        "                                response=result\n",
        "                            )\n",
        "                        ]\n",
        "                    )\n",
        "                )\n",
        "\n",
        "                print(\"Final response:\", final_response.text)\n",
        "    else:\n",
        "        print(\"Direct response:\", response.text)\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    main()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "L_Dla5r6qyrV",
        "outputId": "bc2e9eae-11cc-4095-c822-85edcf7b703e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Enter your currency exchange query: 100 usd to eur\n",
            "Final response: The exchange rate for 100 USD to EUR on the latest date is 88.645 EUR.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Congrats and conclusion**\n",
        "\n",
        "Leveraging function calling via Llama 4 in Vertex AI, you've successfully built a generative AI pipeline that uses the OpenAI and/or Vertex AI SDK! Users can ask about exchange rates, and the system will fetch the latest data from an external API and respond with an answer.\n",
        "\n",
        "Given a prompt from an end-user, Llama takes care of selecting the appropriate function, extracting parameters from the prompt, and returning a structured data object for you to make an external API call.\n",
        "\n",
        "\n",
        "# **Cleanup**\n",
        "You can perform the following cleanup to avoid incurring charges to your Google Cloud account for the resources used in this codelab:\n",
        "*   To avoid unnecessary Google Cloud charges, use the Google Cloud console to  delete your project if you do not need it.\n",
        "*   If you want to disable the APIs for Vertex AI, navigate to the Vertex AI API Service Details page and click Disable API and confirm.\n"
      ],
      "metadata": {
        "id": "pT9Cf6TtAgX7"
      }
    }
  ]
}